Amendments to the Claims: 

This listing of claims will replace all prior versions, and listing of claims in the 
application. 

1 . (Original) A method for structuring video by probabilistic 
merging of video segments, said method comprising the steps of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive 
frames; 

c) extracting a feature set by processing pairs of segments for 
visual dissimilarity and their temporal relationship, thereby generating an inter- 
segment visual dissimilarity feature and an inter-segment temporal relationship 
feature; and 

d) merging video segments with a merging criterion that applies a 
probabilistic analysis to the feature set, thereby generating a merging sequence 
representing the video structure. 

2. (Original) The method as claimed in claim 1 wherein step b) 
comprises the steps of: 

generating color histograms 

generating a difference signal from the color histograms that 
represents the color dissimilarity between consecutive frames; and 

thresholding the difference signal based on a mean dissimilarity 
determined over a plurality of frames, thereby producing a signal that indicates an 
existence of a shot boundary. 

3. (Original) The method as claimed in claim 2 wherein the 
difference signal is based on a mean dissimilarity determined over a plurality of 
frames centered on one of the consecutive frames and corresponding in number of 
frames to a fraction of the frame rate of video capture. 

4. (Original) The method as claimed in claim 2 further including 
the step of morphologically transforming the thresholded difference signal with a 
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pair of structuring elements that eliminate the presence of multiple adjacent shot 
boundaries. 

5. (Original) The method as claimed in claim 1 wherein the 
processing of pairs of segments for visual dissimilarity in step c) comprises the 
steps of computing a mean color histogram for each segment and computing a 
visual dissimilarity feature metric from the difference between mean color 
histograms for pairs of segments. 

6. (Original) The method as claimed in claim 1 wherein the 
processing of pairs of segments for their temporal relationship in step c) 
comprises the processing of pairs of segments for a temporal separation between 
pairs of segments and for an accumulated temporal duration between pairs of 
segments. 

7. (Currently Amended) The method as claimed in claim 1 
wherein step d) comprises the steps of: 

generating parametric mixture models to represent class- 
conditional densities of the inter-segment features comprising of the feature set; 
and 

applying the merging criterion to the parametric mixture models. 

8. (Original) The method as claimed in claim 7 wherein step d) is 
performed in a hierarchical queue and comprises the steps of: 

initializing the queue by introducing each feature into the queue 
with a priority equal to the probability of merging each corresponding pair of 
segments; 

depleting the queue by merging the segments if the merging 
criterion is met; and 

updating the model of the merged segment and then updating the 
queue based upon the updated model. 

9. (Original) The method as claimed in claim 1 wherein 
representing the merging sequence is represented in a hierarchical tree structure. 



-3- 



1 0. (Currently Amended) A computer storage medium having 
instructions stored therein for causing a computer to perform the acts of: 

generating video segments from unstructured video by detecting shot 
boundaries based on color dissimilarity between consecutive frames; 

extracting a feature set by processing pairs of segments for visual 
dissimilarity and their temporal relationship, thereby generating an inter-segment 
visual dissimilarity feature and an inter-segment temporal relationship feature; 
and 

merging video segments with a merging criterion that applies a 
probabilistic analysis to the feature set, thereby generating a merging 
sequence representing the video structure the method of claim 1 . 

1 1 . (Original) A method for structuring video by probabilistic 
merging of video segments, said method comprising the steps of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 

c) extracting a feature set by processing pairs of segments for 
visual dissimilarity and their temporal relationship, thereby generating an inter- 
segment visual dissimilarity feature and an inter-segment temporal relationship 
feature; 

d) generating a parametric mixture model of the inter-segment 
features comprising the feature set; and 

e) merging video segments with a merging criterion that applies a 
probabilistic Bayesian analysis to the parametric mixture model, thereby 
generating a merging sequence representing the video structure. 

12. (Original) The method as claimed in claim 1 1 wherein the 
processing of pairs of segments for visual dissimilarity in step c) comprises the 
steps of computing a mean color histogram for each segment and computing a 
visual dissimilarity feature metric from the difference between mean color 
histograms for pairs of segments. 



13. (Original) The method as claimed in claim 1 1 wherein the 
processing of pairs of segments for their temporal relationship in step c) 
comprises the processing of pairs of segments for a temporal separation between 
pairs of segments and for an accumulated temporal duration between pairs of 
segments. 

14. (Original) The method as claimed in claim 1 1 wherein the 
parametric mixture model generated in step d) represents class-conditional 
densities of the inter-segment features comprising the feature set. 

1 5. (Original) The method as claimed in claim 1 1 wherein step 
e) is performed in a hierarchical queue and comprises the steps of: 

initializing the queue by introducing each feature into the queue 
with a priority equal to the probability of merging each corresponding pair of 
segments; 

depleting the queue by merging the segments if the merging 
criterion is met; and 

updating the model of the merged segment and then updating the 
queue based upon the updated model. 

16. (Original) The method as claimed in claim 1 1 wherein the 
merging sequence is represented in a hierarchical tree structure that includes a 
frame extracted from each segment and displayed at each node of the tree. 

1 7. (Currently Amended) A computer storage medium having 
instructions stored therein for causing a computer to perform acts for structuring 
video by probabilistic merging of video segments, the acts including: 

obtaining a plurality of frames of unstructured video; 

generating video segments from the unstructured video bv detecting shot 

boundaries based on color dissimilarity between consecutive video frames; 

extracting a feature set bv processing pairs of segments for visual 

dissimilarity and their temporal relationship, thereby generating an inter-segment 
visual dissimilarity feature and an inter-segment temporal relationship feature; 
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generating a parametric mixture model of the inter-segment features 

comprising the feature set: and 

merging video segments with a merging criterion that applies a 

probabilistic Bavesian analysis to the parametric mixture model, thereby 
generating a merging sequence representing the video structure the method of 
claim 1 1 . 

1 8. (Original) A method for structuring video by probabilistic 
merging of video segments, said method comprising the steps of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 
detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 

c) extracting a feature set by processing pairs of segments for 
visual dissimilarity and their temporal relationship, thereby generating an inter- 
segment visual dissimilarity feature and an inter-segment temporal relationship 
feature; 

d) merging adjacent video segments with a merging criterion that 
applies a probabilistic Bayesian analysis to parametric mixture models derived 
from the feature set, thereby generating a merging sequence; and 

e) representing the merging sequence in a hierarchical tree 

structure. 

19. (Original) The method as claimed in claim 18 wherein 
representing the merging sequence in a hierarchical tree structure includes 
displaying a frame extracted from each segment. 

20. (Currently Amended) A computer storage medium having 
instructions stored therein for causing a computer to perform probabilistic 
merging of video segments, said instructions performing the acts of: 

a) obtaining a plurality of frames of unstructured video; 

b) generating video segments from the unstructured video by 

detecting shot boundaries based on color dissimilarity between consecutive video 
frames; 
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c) extracting a feature set by processing pairs of segments for 

visual dissimilarity and their temporal relationship, thereby generating an inter- 
segment visual dissimilarity feature and an inter-segment temporal relationship 
feature; 

d) merging adjacent video segments with a merging criterion that 

applies a probabilistic Bayesian analysis to parametric mixture models derived 
from the feature set, thereby generating a merging sequence; and 

e) representing the merging sequence in a hierarchical tree 

structure the method of claim 18. 



2 1 . (New) A method for structuring video by probabilistic 
merging of video segments, said method comprising: 

generating video segments from an unstructured plurality of video 
frames by detecting shot boundaries based on color dissimilarity between 
consecutive frames; 

extracting a feature set by processing pairs of the video segments 
for visual dissimilarity and processing pairs of segments to determine temporal 
separation; and 

merging the video segments with a merging criterion that applies a 
probabilistic analysis to the feature set, thereby generating a merging sequence 
representing the video structure, the merging independent of any empirical 
parameter determination; and 

generating a hierarchy with the merged video segments, the 
hierarchy having a merging sequence represented by a binary partition tree. 

22. (New) The method as claimed in claim 21 wherein the 
merging the video segments includes maximizing the a posteriori probability mass 
function of a binary random variable that represents inter-segment features of the 
video segments. 
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